Repository navigation
fix(skills): name optional Glean tools with a fallback - #51
Draft
steve-calvert-glean wants to merge 2 commits into
Draft
steve-calvert-glean wants to merge 2 commits into
steve-calvert-glean wants to merge 2 commits into
Conversation
A Glean MCP server's tools/list is the only complete list of its tools, and it differs between deployments: admins add and remove toolsets, and some deployments serve a tool under another name. Skills listed tools such as code_search, meeting_lookup and the email tools as if every deployment had them. Skills now name search, read_document, chat and employee_search freely, and name any other Glean tool only with a fallback for when it is missing. using-glean says that tools/list is the source of truth, splits its routing table into core tools and tools to use if available, and drops the host-specific mcp__glean tool prefix. It also gains rules to run independent calls in parallel, stop once the question is answered, retry a search at most twice, and say when sources are stale or disagree.
Add scripts/check-skill-tool-names.mjs (npm run check:skill-tools) and run it in CI with its tests (npm run test:scripts). It fails when a skill names a non-core Glean tool without saying the tool may be missing, or uses a host-specific mcp__glean prefix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A Glean MCP server's
tools/listis the only complete list of the tools it serves, and that list differs between deployments. Admins add and remove toolsets, and some deployments servesearchasenterprise_search. Our skills hardcoded tool lists anyway:using-gleanrouted questions tocode_search,gmail_search,outlook_search,knowledge_graph_queryand others as if every deployment had them, andagents-as-tools.mdkept a fixed list of 13 "built-in" tools. A hardcoded list can't stay in sync with a live server.This also replaces #49, which added a second, shorter Glean skill that triggered on the same questions as
using-glean.Change
search,read_document,chatandemployee_searchfreely. These are on nearly every deployment.code_searchif available; otherwisesearch".using-gleansays thattools/listis the source of truth and that the tools it names are examples. Its routing table is split into core tools and "other tools, if available", each with a fallback. The host-specificmcp__glean_<server>__<tool>naming is gone.using-gleangains the rules from feat: add /glean skill for cited company answers at minimal tool cost #49 that weren't already there: run independent calls in parallel, stop once the question is answered, retry a search at most twice, and say when sources are stale or disagree.scripts/check-skill-tool-names.mjsenforces the rule in CI (npm run check:skill-tools, with tests innpm run test:scripts). It found 89 problems in 21 files onmain.How the check decides
Only tool names in backticks count. A non-core tool passes when a sentence that names it states a condition ("if
xis available", "when it is in your tool list"), when it sits under a heading that says "if available", when the file's opening text names it with a condition, or when the file is that tool's own reference file. The core set and the list of known tools are constants at the top of the script.Testing
npm test(build, thenpluginpack validatefor claude, cursor and codex)npm run check:skill-toolsnpm run test:scripts(14 tests)Open question
The plugin's relay exposes 8 tools (
REMOTE_TOOLS_ALLOWLIST), andcode_search, the email tools and the knowledge graph tools are not among them. Through the plugin, those skills now fall back tosearch. Whether the plugin should expose those tools is a separate decision.